The best books on pattern recognition and how we make sense of our random world

The most basic, the most life-determining story is the one you tell yourself about yourself. Who are you? How did you get to be that way? And by the way, what is reality?

I reviewed this book for The Wall Street Journal and found it eye-opening. I was struck by Berns’s account of “how other people’s narratives worm their way into our brains,” creating a consensus version of reality he calls “a shared delusion.”

This delusion can become self-limiting, even self-destructive. And so we have counterfactuals, the narratives of possibiity that allow us to escape these constraints—to change our story, if we have the will to do so.

Obviously, some of us are more aware of patterns than others. Simon Baron-Cohen—a psychologist at Cambridge, and one of the world’s leading authorities on autism—has found that a facility for pattern recognition is strongly correlated not only with gender (males predominate) but with autism.

He led a survey of 600,000 Britons aimed at determining if they were primarily empathizers, adept at connecting with other people, or systemizers, more interested in detecting patterns and learning how things work. Those at the extreme end of systemizing were considerably more likely to be autistic.

Baron-Cohen’s empathizer/systemizer questionnaire is included at the back of the book. Taking the bait, I found myself on the cusp of extreme. Which may explain a lot.

Gladwell became famous for uncovering patterns in the social behavior of humans; I wanted to find the patterns—some would say formulas—that characterize his books. Why are these books so popular? Why are critics so disdainful?

Here he revisits his first, a huge bestseller, and finds that the model of viral contagion that seemed hopeful and bright a quarter-century ago now looks dark and foreboding.

What makes both books work is a combination of diligent reporting, narrative sleight-of-hand, and a nose for patterns. “There must be a set of rules,” he writes, “buried somewhere below the surface.” No, his rules don’t always hold. But our need for patterns that explain the chaos around us has a lot to do with his success. 

This book addresses the key problem with pattern recognition: Why do we see patterns that don’t exist? And why are these patterns so often dark?

We assume that what our eyes see is actually there, and what our brains comprehend is real. Not necessarily so, but try telling that to your brain.

Confirmation bias leads us to filter out facts that challenge what we think. In any dispute, no matter the evidence, each side remains convinced it’s right and the other side is delusional. And because the brain always wants an explanation, it’s easy to conclude that there’s a secret set of rules, maybe even a secret cabal that enforces them.

Brotherton argues convincingly that conspiracy theories aren’t restricted to a bunch of paranoid kooks; they’re just a function of being human.

Harari cuts to the chase. Who else would try to tell the history of humankind, as he did in Sapiens, in just over 400 pages?

His trick is mapping patterns to narratives: Religion is a story; nations are a story; money is a story. 

And information networks carry those stories. Such networks long predate the internet, but social media algorithms have distorted their function, spreading outrage in order to maximize user engagement and increase profits.

Because AI can think and act on its own, Harari writes, it “changes the fundamental structure of our information network.” Plus, it can process vast data streams and spot patterns we can’t. So why would we hang onto something we’re not that good at? Maybe because it wouldn’t be smart to cede control of something so basic to our existence.